Real-time 3D Reconstruction on Construction Site using Visual SLAM and UAV

3D reconstruction can be used as a platform to monitor the performance of activities on construction site, such as construction progress monitoring, structure inspection and post-disaster rescue. Comparing to other sensors, RGB image has the advantages of low-cost, texture rich and easy to implement that has been used as the primary method for 3D reconstruction in construction industry. However, the imagebased 3D reconstruction always requires extended time to acquire and/or to process the image data, which limits its application on time critical projects. Recent progress in Visual Simultaneous Localization and Mapping (SLAM) make it possible to reconstruct a 3D map of construction site in real-time. Integrated with Unmanned Aerial Vehicle (UAV), the obstacles areas that are inaccessible for the ground equipment can also be sensed. Despite these advantages of visual SLAM and UAV, until now, such technique has not been fully investigated on construction site. Therefore, the objective of this research is to present a pilot study of using visual SLAM and UAV for real-time construction site reconstruction. The system architecture and the experimental setup are introduced, and the preliminary results and the potential applications using Visual SLAM and UAV on construction site are discussed. INTRODUCTION In recent years, 3D reconstruction and mapping using unmanned aerial vehicle (UAV) has gained a lot of interest in civil and construction industry. Compare to ground vehicles, UAV are quicker, safer and more cost effective (Gheisari and Esmaeili). It can easily reach areas that are inaccessible to manned vehicles and undertake tasks that are dangerous to humans. Currently, the primary sensing technique equipped on small UAVs is photogrammetry. Compared to other remote sensors, such as LiDAR, radar and ultrasound, photogrammetry relies on optical sensor that has the advantages of texture rich, low-cost and light weighted, and can satisfy the required level of accuracy for most applications (Dai et al. 2012). With onboard cameras, UAV can take aerial images during flight and produces 3D point clouds through image-based postprocessing techniques (Liu et al. 2014). Recent applications showed the potentials of using UAV to monitor construction progress (Lin et al. 2015), survey civil infrastructures (Chan et al. 2015), and document historical site (Themistocleous et al. 2016). However, due to the heavy computational workload of image processing and computer vision algorithms, the model generation process is still slow (Nex and Remondino 2014), which limits its implementation on time critical applications. Simultaneous localization and mapping (SLAM) is an advanced technique in robotics community which was originally designed for a mobile robot to consistently build a map of an unknown environment and simultaneously estimates its location in this map (Durrant-Whyte and Bailey 2006). When camera is used as the only exteroceptive sensor, such technique is called Visual SLAM or VSLAM (Artieda et al. 2009). Similar as photogrammetry, Visual SLAM has the advantages of rich visual data and low-cost. Previous applications of Visual SLAM and UAV mainly focus on indoor scanning and mapping (Michael et al. 2012; Scherer and Zell 2013), and only little or no effort has actively tested this technique at outdoor construction site (Ham et al. 2016). For many real-world applications in construction industry, timely data input play a key role of project success. Therefore, the objective of this study is to fill this knowledge gap by investigating the feasibility of using visual SLAM and UAV for real-time scene reconstruction and 3D mapping on construction site. The following paragraphs are organized as follows: In section 2, related works of 3D reconstruction using optical sensors and UAV are summarized. In section 3, the general workflow of visual SLAM is briefly demonstrated. In section 4, the system architecture applied in this study, which includes both the hardware and software configuration, is introduced. In section 5, the experimental setup for outdoor environment using the proposed system is presented. In section 6, the authors evaluated the accuracy of SLAM generated model and presents the preliminary results of three potential applications using Visual SLAM and UAV on construction site. Finally, the conclusion, limitation and future study are discussed. LITERATURE REVIEW In civil and construction industry, most image-based 3D models are developed with post-processing techniques, which is a cascaded collection of image processing and computer vision algorithms (Lowe 2004; Triggs et al. 1999). With such technique, early investigators evaluated the availability of using UAV and onboard camera to survey historical site for landscape and heritage documentation (Remondino 2011; Remondino et al. 2011). 3D reconstruction on working construction site can be used to estimate excavation volume and track earthwork process (Siebert and Teizer 2014). The excavation volume was measured by generating digital elevation model (DEM) and the earthwork progress was tracked by monitoring the change of cut and fill area during earthmoving and compacting activities. Integrating the image generated 3D model with 4D BIM, the performance of UAV-enabled construction process monitoring can be increased, objects such as on-site occlusions can be identified and removed from camera point of view, and the ideal paths of UAV can be pre-determined (Lin et al. 2015). The image-based 3D model is also able to assist in condition assessment of in-service building and civil infrastructures. A recent application reconstructed an accurate model of a large-scale timber truss bridge using UAV and onboard camera (Khaloo et al. 2017). The model generated based on a total of 22 flights with more than 2000 high resolution images. By comparing with LiDAR, the result showed that although the noise floor of image-based 3D model is nearly three times higher, the density of points is much larger, which shows a better performance on structural detail representations. Other related applications of image-based postprocessing 3D modeling using UAV includes subsidence surveying in abandoned mine areas (Suh and Choi 2017), terrian mapping (Stumpf et al. 2013), disaster site reconstruction (Ferworn et al. 2011) and forest investigation (Wallace et al. 2012; Zarco-Tejada et al. 2014). Compare to the offline modeling applications, in construction industry, only limited efforts have been found on real time image-based 3D reconstruction using UAV. Previous study generated 3D point cloud of building façades by mounting a Kinect sensor on a hovering UAV (Roca et al. 2013). The Kinect range sensor provides both visual and depth image that can generated 3D colored point cloud in real time. However, due to the limitations of infrared sensors, the study fails to reconstruct objects with highly reflective materials or under extreme light conditions. In (Michael et al. 2012), a novel experiment was carried out to reconstruct the layout of a earthquake-damaged multistory building using both a ground robot and a small UAV. The ground robot equipped with an onboard rotating laser scanner provides feature-rich point cloud, the UAV attached both a laser scanner and a Kinect sensor take over the mapping tasks at areas where the ground robot cannot access. This study presented a potential of combining UAV and Visual SLAM for efficient 3D reconstruction and mapping on construction site. WORKFLOW OF VISUAL SLAM Although different algorithms of Visual SLAM existed, most studies follow the general workflow as feature extraction and matching, refinement of matching errors, loop closure detection and global map optimization (Fuentes-Pacheco et al. 2015). Compare to monocular camera, Visual SLAM with stereo camera provides both visual information and depth stream that increases the robustness of real-time mapping (Yousif et al. 2015). In this study, the authors implement a RGB-D graph-based SLAM approach introduced by (Labbe and Michaud 2013; Labbe and Michaud 2014) on a UAV. The approach applies appearance-based loop closure detection method, which uses bag of words, to associate the new images with the previous frames, and generate a global map with graph pose optimization. A three-layer memory management mechanism is designed to attenuate the accumulated memory usage for large-scale, long term, and multi-session mappings. Figure 1 shows the general workflow of using an RGB-D sensor and the graph-based SLAM approach for 3D reconstruction of an indoor office. Figure 1. Workflow of generating the 3D map of an indoor office using RGB-D camera and the graph-based SLAM approach